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METHOD AND APPARATUS FOR 
ALLOCATING BANDWIDTH IN 
TELECONFERENCING APPLICATIONS 
USING BANDWIDTH CONTROL 

BACKGROUND OF THE INVENTION 

Teleconferencing systems among computer systems or 
work stations have become a common means for exchanging 
ideas among work groups of people located in different 
locations. Teleconferencing systems mimic actual meetings, 
in that the human users can see one another and talk to one 
another in real-time, without having to type in their mes- 
sages into a keyboard or display responses as text on a 
monitor or a printer. Teleconferencing also allows multi- 
party communication^ both in multi-directional conversa- 
tions and in broadcast from one location to many locations. 

A network may be a local network connecting a few 
machines to one another, or a much wider network connect- 
ing large numbers of different types of machines. Many 
networks, especially wide area networks, connect machines 
operating on different platforms, but provide consistent 
protocols to allow the machines to communicate. Various 
approaches to networking are known in the art, including 
distributed networlss and centrally administrative networks. 

FIG. 1 shows one example of a network. Processors 150C 
and 155C are each connected via a network adapter 160 and 
165, respectively, to a network medium 170. The network 
medium 170 may be a digital bus, a video coaxial cable, a 
telephone line, a fiber optic cable, or any other medium 
through which information may be trausferred from one 
location to another. It will be understood upon reference to 
FIG. 1 that other arrangements are possible, that the network 
may include more than two computer systems, and that each 
of the processors 150C and 155C may be connected via 
other network adapters to other network media. 

Each of the processors in the computer systems shoAvn in 
FIG. 1 has a video monitor at 150D and 155D, a video input 
150A and 155A, an audio input 150B and 155B, a keyboard 
input 150e and 155^ having a mouse, and possibly other 
peripheral input output devices connected thereto. It will be 
understood that each computer system may also comprise 
multiple processors sharing a network adapter, forming a 
local network within the larger networic shown in FIG. 1, 
Computer systems such as 150 and 155 may connect to a 
number of network media having differing types of media 
substrates, and further having different network protocols. 
Processor 150C and 155C each display images on the video 
monitor 150t) and 155D, respectively, and receive inputs 
from other peripherals. Processors may also be running 
computer programs, including application programs and 
transport layer programs, that may call one another and 
serve one another, exchanging data, addresses, and control 
signals. 

One embodiment of a possible computer system operating 
according to one embodiment of the present invention, is 
shown in FIG. 2. A processor 302 is connected via a system 
bus 301 to a main memory 304, a read only memory 306, 
and a mass storage device 307. The main memory may be a 
volatile memory array composed of dynamic random access 
memory. The read only memory 306 may be composed of a 
CD ROM, an initialization cache, erasable programmable 
read only memory, EEPROM, flash memory, or other read 
only memories. The mass storage device 307 may be con- 
figured as a disk drive writing to, and reading from, hard 
disks, floppy disks, or other storage devices. Processor 302 
may also have a cache, either a write back or read through 
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configuration, storing frequently used values in a static 
random access memory or other memory array, the cache in 
some configurations being coupled directly to main memory 
304. Various other intelligent cards may be connected to the 

5 bus 301, including direct memory access devices. 

Also shown in FIG. 2, various peripherals exchange 
information via bus 301 with the processor 302, main 
memory 304, read only memory 306, and mass storage 
device 307. These peripherals include a display 321, and 

jQ frame buffer 341, generally a video monitor or printer 329. 
A keyboard 322 is also coupled to the bus 301, permitting 
alphanumeric entry. A cursor control 323 coupled to the bus 
301 may be configured as a mouse or track ball. A sound 
output device 321, also coupled to the bus 301, may be 

25 configured as a loud speaker or a number of loud speakers, 
A video input device 325 may be configured as a video 
camera, a scanner, a fax input, or similar device, and is 
coupled to the processor 302 via bus 301. A sound input 
device 326, also coupled to the bus, may be configured as a 

2Q microphone or a sound synthesizer, or may be a telephone 
connector. A network or coDomunication medium connector 
342 allows communication between any of the above 
devices and a network or communication medium. Finally, 
a communication device 327, also coupled to the bus 301, 

25 allows communication between any of the above devices 
and the network (or other communication) medium 170 via 
the network/communication adapter 160, In some 
embodiments, the networking medium is a digital commu- 
nication medium, allowing high speed digital commuuica- 

3Q tion between computer systems over the network. 

Various network configurations are known in the art: star 
configurations, daisy-chained networks, etc. The computer 
system, in some embodiments, is connected via a network 
adapter to a network medium that allows the computer 

35 system to communicate with other computer systems at 
other locations. The other systems may be coupled to the 
same medium substrate, or may be connected to other 
processors or intelligent devices that are in turn cotmected to 
the substrate. 

40 A processor or processors of the computer system as used 
in one embodiment of the present invention generally has 
software operating thereon, including teleconferencing 
applications. Data is transferred over the network by the 
teleconferencing applications, including but not limited to, 

45 voice, video, and other data. The data may have been created 
by the same application that created it, or may have been 
created by another application. Other applications may also 
be operating on the computer system, sending or receiving 
documents, for example. Also, other teleconferences may be 

50 ongoing at the same time, each supported on the same 
network. When several teleconferences are ongoing at any 
given time over the same network, it may be that each 
includes a different subset of the sets of computer systems 
connected to the network- Teleconferencing applications 

55 often also allow other computer systems to join ongoing 
teleconferences by selecting which teleconference a user at 
the other computer systems wants to call, or drop out of 
teleconferences by terminating connections. 
FIG. 3 illustrates a hierarchy of programs or of levels of 

60 abstraction in a single program, allowing an application to 
communicate over a network. The application program 401 
includes a presentation layer and a human user interface for 
exchanging information with a human user. Various appli- 
cations programs may call software at conferencing layer 

65 400 that has its own protocols for exchanging information 
with the application and with the lower layers of the com- 
munication system. The conferencing layer software used by 
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a particular application program parses the information 
exchanged with the human user into messages provided to 
one or more transport components 402. Each transport layer 
exchanges information with the conference layer, and pro- 
vides packets of information suitable for a particular net- 
work component 403. The network layer exchanges infor- 
mation with the transport layer, and provides data in a form 
appropriate through a system network component to the 
network adapter 160. 

Among the applications that may be run on a processor 
are teleconferencing applications. Teleconferencing applica- 
tions generally allow a user to view video imageS;, send 
video images, and/or send and receive audio over a network, 
generally in real time. Real-time requires not only data 
transmission of a fairly large amount of data, but also 
requires a high degree of synchronization among the data 
when packets are reconstructed by a receiver system; if a 
voice, for example, arrives before a video image of a person 
speaking, the sound and video must be synchronized when 
presented by the application running on the receiver system. 
Voice (or other sound), video, and other signals, including 
control signals, must be presented to the higher-layer at a 
proper timing to have the proper meaning. 

Problems arise, however, when a large number of appli- 
cations on a given network node generate signals for trans- 
mission across a network or other communication medium. 
For example, when a particular computer or workstation is 
running several teleconferencing applications 
simultaneously, or running several other applications that 
similarly require network bandwidth, the applications vie for 
access to the network against one another. It will be under- 
stood that "network" refers generally to any shared com- 
munication pathway from one system to another, and spe- 
cifically to a shared communication pathway of limited data 
capacity in terms of bandwidth or data rate. Generally, each 
layer of software, including the transport layer and other 
lower level communication layers, like the hardware, have 
bandwidth restrictions limiting the amount of information 
that may be provided from any given system to the network 
in a given period of time. Therefore, applications compete 
for output bandwidth against one another, and in many 
systems the first to claim a bandwidth can use that band- 
width to the exclusion of other applications. 

The problem is especially acute when a teleconferencing 
application is used, in that teleconferencing applications 
require such a large bandwidth commitment, and generally 
lack flow control that would limit the flow of data from the 
application to the network when the network is heavily used. 

Sharing bandwidth fairly is a practical concern. A network 
may include any number of systems. A network may also 
include multiple simultaneous teleconferences. A particular 
system on the network may be involved in more than one 
simultaneous teleconference; the teleconferences may be 
running under different applications, or under different cop- 
ies of the same teleconference. Multiple applications or 
other processes on a computer system may have access to a 
network or other communications medium. Any given tele- 
conference may require a system to provide teleconferenc- 
ing messages to any number of the other systems on the 
network. The simultaneous teleconferences at a given sys- 
tem may run under different teleconferencing applications, 
compression schemes, and update rates, and the different 
messages may be addressed to different nodes on the 
network, and to any number of nodes. All of these increase 
the use of bandwidth by a system. 

The number of recipients of a particular message, sent 
over the network from a particular system, in the context of 
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a particular teleconference, running in a particular applica- 
tion or copy thereof, is the "fan out" of the teleconference 
message. When the number of parties to a teleconference 
becomes very large, the fan out becomes very large as well. 
5 Ulis can burden the network, since each teleconference 
within each application may require a separate message to 
be sent to each other system participating in the teleconfer- 
ence. Some teleconferencing applications do not have 
"multi-cast", which is a feature that allows a single message 
to be received by multiple nodes on a network, but some 
teleconferences must send a separate message to each other 
participant in a teleconference, imposing an enormous bur- 
den on the network transport layer programs when the 
number of nodes participating in the teleconference is high, 
and when several teleconferences or other network-using 

^5 applications are ongoing simultaneously. 

Excessive bandwidth use by any given connection pre- 
sents another problem. Teleconferencing systems and other 
network bandwidth users exist that expand to fill all avail- 
able bandwidth. For example, teleconferencing systems may 

20 send video updates as frequently as is possible given a 
particular bandwidth availability. Some flow control 
schemes assign bandwidth on a first-come, first-served 
approach among the applications, and later-initiated network 
applications must wait in a "queue" until earlier applications 

25 have completed transmission or receipt of the network 
message. Because teleconferencing applications generally 
do not have flow control while other network applications 
frequently do, the non-teleconferencing applications can be 
slowed down tremendously when teleconferencing is in use. 

30 Other network transmission applications, for example, fax- 
ing of documents, can compete with the video or audio 
teleconferencing systems for bandwidth, and together with 
such systems can occupy all available bandwidth. Most 
non-teleconferencing applications have an internal flow con- 

35 trol that identifies bandwidth limits and reduces data rate 
accordingly; however, the methods used are not appropriate 
for real time teleconferencing applications. Such other file 
transfers occupy bandwidth as well. When an aggressive 
application sends excessive numbers of video update frames 

40 or otherwise occupies bandwidth that can better be allocated 
to other applications, the other applications (such as file 
transfer, faxing, etc.) slow the rate at which the non- 
teleconferendng applications send data. Teleconferencing 
applications generally do not have flow control features, and 

45 so provide data to a network or communication/transmission 
medium at an rate that is less predictable. Teleconferencing 
applications can easily slow down other applications to an 
almost stand-stin rate. 
Application-specific solutions are not enough to solve the 

50 problem. Even node-specific solutions cannot eliminate the 
problem. Some application-specific flow control processes 
switch to better compression algorithms, transmit fewer 
updates, transmit sound in mono instead of stereo, require a 
smaller video window, or otherwise reduce bandwidth con- 

55 sumption when bandwidth is at a premium. However, these 
systems only affect bandwidth use by the application itself, 
and do not accommodate other applications. 

Error-correcting protocols are not appropriate for real- 
time applications. These protocols are useful in correctly 

60 transmitting files having exact information, such as a spread- 
sheets or other documents, in which small errors must be 
corrected and re -transmissions do not create problems, and 
so are commonly available among transport layer programs. 
Error-correcting protocols, such as most transport-layer 

65 protocols, are inappropriate for conferencing or other real- 
time applications. Error correcting protocols require 
re-transmission of erroneous packets of data 
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Therefore, it is desirable for a system to be found that can 
allocate bandwidth among applications to prevent one appli- 
cation from excluding other applications from the network. 
Also, because an application running on one workstation or 
computer can reduce bandwidth available to other computer 5 
systems and to other users when systems share access to a 
common network or other communication medium, it is 
desirable to have a central control for administration of the 
bandwidth allocation to prevent an application from inter- 
fering with other applications. It is also desirable to protect lo 
the bandwidth allocation from manipulation at the worksta- 
tion itself, to prevent users from changing the allocation 
once it is established. It is also desirable to allow network 
management process to manipulate or alter the variables as 
needed. 15 

SUMMARY 

In the present invention, in some embodiments, an admin- 
istrator assigns a total bandwidth allocation to at least one 
other computer system, and the computer system parcels the 20 
bandwidth among the applications ranning on the computer 
system. In the operation of one embodiment of the present 
invention, an administrator sends a bandwidth maximum 
allocation to each node on the system. Each node determines 
a current bandwidth being used, and limits the current 
bandwidth to this allocation. Thereafter, each node then 
reallocates its usable bandwidth among applications running 
on the nodes that are attempting to send messages over the 
network. For each application, a current bandwidth use is 
determined, as well as a current bandwidth demand. The ^ 
current bandwidth demand is the amount of bandwidth that 
the application would be usii^ if no other applications were 
nmning on the node and if there were no limitations on the 
amount of data the application could send to the network. A 
ratio is calculated to determine the amount of the bandwidth 
demand currently being satisfied for each application, thus 
calculating the happiness factor ifor the appUcatton. 

FIELD OF THE INVENTION 

40 

The present mvention relates to computer networks, and 
more specifically to network administration. 

BRIEF DESCRIPTION 

FIG. 1 shows one example of a network. 45 

FIG. 2 shows one embodiment of a possible computer 
system operating according to one embodiment of the 
present invention. 

FIG. 3 illustrates a hierarchy of programs or of levels of 
abstraction in a single program, allowing an application to 50 
communicate over a network. 

FIG. 4 shows a managed network of nodes comprising 
work stations or other computer systems and permitting a 
number of teleconferences to operate over the network 
among the nodes. 55 

FIG. 5 shows BlueDotUsingBandwidth as implemented 
according to one embodiment of the present invention. 

FIG. 6 shows Update Values as implemented according to 
one embodiment of the present invention. 

FIG. 7 shows BlueDotGimmeBandwidth as implemented 
according to one embodiment of the present invention. 

FIGS. 8A, 8B, and 8C show Update allocations as imple- 
mented according to one embodiment of the present inven- 
tion. 65 

FIG. 9 shows BlueDotQueryHappiness as implemented 
according to one embodiment of the present invention. 
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FIG. 10 shows a memory array storing steps to be 
perfonned at the client (non -administrator) node. 

FIG. 11 shows a memory array storing steps to be 
performed at the administrator node. 

DETAILED DESCRlFnON 

In the present description, the terms "computer system", 
"work station", "machine", and "node" will be used inter- 
changeably so as not to obscure the present invention 
needlessly. It will be understood that any of these may be 
replaced with a plurality thereof, or may be replaced with 
other intelligent systems such as servers, remote dedicated 
machines. Computer systems, dedicated servers, work 
stations, and other machines may be connected to one 
another across networks. For simplicity of explanation, the 
terms "machine", "computer system", 'Vork station", and 
"node", will be used interchangeably, although it will be 
understood that each may refer to a single intelligent system 
or to a distributed sub-network sharing a network adapter to 
the network administrated by the described network admin- 
istrator. 

A portion of the disclosure of this patent document 
contains material which is subject to copyright protection. 
The copyright owner has no objection to the facsimile 
reproduction by anyone of the patent disclosure, as it 
appears in the Patent and Trademark Office patent files or 
records, but otherwise reserves all copyright rights whatso- 
ever. Copyright Apple Computer, Inc. 

In a network-level aspect of the present invention, a client 
system receives a value from an administrator node and 
determines the bandwidth allocation. 

A network administrator node is a computer system on a 
network that controls network access of other nodes on the 
same network. A network administrator node must exchange 
information with the other nodes on the network, however, 
to ensure that higher priority uses of the network bandwidth 
receive a greater access than those with lower priority uses. 
However, the network administrator must also be able to 
limit network access by aggressive though low priority 
applications, or a low priority application use might capture 
the network, preventing higher priority users from gaining 
access. Also, within each computer system, various intelli- 
gent components must share access to the network adapter, 
often requiring local arbitration of network demands. 

Although local arbitration may be performed by the 
computer system itself, a central network administrator in a 
centrally administrated network prevents usurpation of the 
network bandwidth by limiting network access of each 
computer system on the network. The administration of the 
network is flexible, however, in that a read only variable 
provided to the network by the computer system informs the 
administrator of the bandwidth of the messages the various 
programs and processors are trying to put on to the network. 

The applications transfer data by calling other lower-level 
programs on the computer system that convert the data into 
a more easily transferred form. Standard forms are packets 
of data, that may be of any size. Packets of data are 
transferred to the network from the computer system via a 
transport layer protocol specific to the network medium and 
to the computer system. 

FIG. 4 shows a managed network of nodes comprising 
work stations or other computer systems and permitting a 
number of teleconferences to operate over the network 
among the nodes. Systems 501-517 are individually coupled 
to one or more network media, allowing communication 
over the network. Some systems 503, 505 appear on more 
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than one network, and may allow comnaunications between in more than one teleconference (or other application requir- 
one network and another. For example, system 503 can ing sending messages to the network), the allocated band- 
provide data to, and receive data from, network or commu- width is further allocated among the various teleconferenc- 
nications media 530 and 532, or can facilitate exchange of jng and other applications attempting to send messages to 
data between the nehvork or communications media 530 and 5 the network. In one embodiment of the present invention, 
532, Other systems 501, 507-517 appear on only one the network administrator monitors the nodal happiness 
network. factors set as a read only variable by each node, and can 
On any given network or communications medium, one adjust the maximum bandwidth allocation per node accord- 
or more admmistratorsmay be present, assigning allocations i^gjy Although users at each node might change the tele- 
of bandwidth to each of the nodes (computer systems) on the ,0 conferencing application or other^^dse demand additional 
network or communications medium For example, system bandwidth, for example, by turning off data compressors or 
501 can be assigned a maximum bandwidth allocation by a transmitting in stereo, the node cannot reset or alter the 
network administrator. In some embodiments, a network maximum bandwidth allocation determined by the network 
administrator allocates bandwidth immediately; in other administrator 

embodiments, the administrator waits until bandwidth use of -,c t it. , , . . . , 

the network or communications medium reaches a prede- one embodiment, a ne^^ork administrator pubhshes 

termined value before aUocating maximum bandwidth alio- J'^u .u u ^ ^^ ? ul*"' 

cations. In some embodiments, onlv some of the systems are 2!^^°* client nodes publ^h three other variables, 

assigned a maximum bandvvadth allocation, while others are ^^^^^^^ ^^"f determined at each node, 

not; in other embodiments, each system is assigned a maxi- ^ ^ach system receives the value determined for the 

mum bandwidth allocation, though the allocations are not ^^^"^^^ admimstraton The four variables are: 

uniform throughout the cluster of computer systems coupled current total bandwidth bemg used for all conferenc- 

to the network or communications medium. As will be functions on each particular machine, measured in 

explained below, the administrator receives certain values embodiment in bytes per second; 

from the other systems coupled to the same network or 25 number of active applications that are capable of 

communications medium, and these vahies can change over sending messages over the network in the immediate 

time, so that the allocations can also change. For example, future, a value that may be zero if the particular 

a system may discontinue transmission of data, or higher- machine is merely receiving (and not sending) mes- 

priority users may begin. As will also be explained below, sages; 

processes running on the client (that is, non-administrator) 3^ a maximum allowed bandwidth per node, assigned by the 

nodes also receive values from the network or communica- administrator; 

tions medium, the values including the maximum bandwidth ^he number' of active connections on the node, 

allocation. At each chent node that receives such an aUoca- ^he variables can contain values pertaining to any group 

tion value, either aU applications or a group of apphcations bandwidth users, although in the preferred embodiment, 

ma selected class are hmitedm their abihty to send data to 35 group of bandwidth users consists of the group of 

the ne^ork or communications medium, the limit imposed teleconferencing applications. It will be apparent, however, 

by various embodiments of the present mvention. ^he group of aU applications, any group of processes 

The present invention mcludes m one embodiment a operating below (or above) the level of applications, or even 

means for a network administrator to control the bandwidth hardware could have been selected, 

used by each system node in a distributed network, and 40 With the exception of the allowed bandwidth per node, the 

further allows each node to allocate among the appUcations values are determined by each non-administrator node and 

running thereon the bandwidth in a feir manner. The network provided to the administrator via the network. The admin- 

administrator can alter the bandwidth allocation among the istrator node thus monitors: total bandwidth use by each 

various workstations, servers, or other processing nodes in node, the number of active apphcations currently transmit- 

the network, thus reallocating bandwidth among the nodes. 45 ting (that is, sending) data over the network from each node. 

The network includes an administrator node, operated by and the number of connections at each node, 

a network administrator. The network administrator may be The number of connections at any given node may be 

controlled by a person, or by a computer program. Other considered a total of the fan out values taken over all active 

embodiments allow some management functions to be applications. As stated above, the fan out is the number of 

shared by two or more processes on the manager node. 50 separate copies ofteleconferendng data that must be sent to 

Although the fimctions of the administrator may in some the network or commumcations medium due to the number 

embodiments be performed by two or more manager nodes, of receivers participating in the teleconference. If a feature 

in general the described embodiment includes only one called "multi-cast" is available, then all addressees can 

administrator performing all network administration func- receive the same copy, and only one need be sent, and the fan 

tions. Various signals are communicated from each non- 55 out is one. Ifmulti-cast is not available, then a separate copy 

administrator system over the network to the administrator, is sent to each recipient, and the fan out is the number of 

and signals are received from the administrator. According recipients. If several applications are running on a computer 

to these signals, the described embodiment of the present system or if the computer system is engaged in several 

invention locally manages network bandwidth used by teleconferences, the fan out of each is totaled to derive the 

applications operating thereon. eo number of connections at the node. This total number of 

A network administrator detects the additional network connections is provided to the network, 

bandwidth demand, and, according to one embodiment of In response to these values from the various non- 

the present invention, sets a bandwidth maximum allocation administrator nodes, the administrator assigns a maximum 

for each node. If one teleconference requires greater band- bandwidth to each node. The network administrator can also 

width than another, the netv^'ork administrator can set the 65 determine a priority for bandwidth use among applications, 

bandwidth allocations differently for the nodes participating or can allow system processes on each node to determine 

in the different teleconferences. At the nodes that participate priority. The maximum bandwidth is provided over the 
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network to each of the non-administrator nodes. The admin- 
istrator is not limited in bandwidth allocation; for example, 
maximum bandwidths need not be uniform across a net- 
work. Different nodes have different bandwidth needs and 
different priorities. Some nodes, for example ATM switches, 5 
have very large bandwidth availability; other nodes may be 
daisy chained via an ethemet connection to a large network 
of nodes, each of whom is contending for bandwidth. Also, 
if a network is distributed into segments interconnected via 
routes, local boxes, or ethemet boxes, each node may be lO 
given a larger bandwidth allocation, since nodes in each 
segment may be sending simultaneously. Because of these 
and other non-uniform requirements, different nodes can be 
allocated different bandwidth allocations. 

Because only the administrator has access to the maxi- 15 
mum bandwidth, only the administrator can change the 
maximum allowed bandwidth of any machine on the net- 
work. Without this access, no machine can change even its 
own maximum allowed bandwidth. 

Occasionally, a machine runs several conferencing appli- 20 
cations having different fan outs and may also be sencKng 
files or faxing documents over the network, sometimes to 
other nodes involved in a particular conference, and some- 
times to other nodes. As indicated above, however, the 
maximum bandwidth allocated by the administrator applies 25 
only to the selected group of applications, e.g. in the 
described embodiment, only to teleconferencing applica- 
tions. Thus, only teleconferencing applications are 
bandwidth-limited by this embodiment of the present inven- 
tion; other applications can use bandwidth unrestrictedly. 30 

Computer systems on a network may use less than all the 
bandwidth assigned to them. Although an assignment of 
bandwidth from the administrator serves as a maximum, the 
current bandwidth being used may be far less than the 
assigned maximum; in fact, the current bandwidth being 35 
used may be zero, for machines that are receiving data 
without sending data over the network. 

In a system-level aspect of the present invention, the 
client processor reacts to the maximum bandwidth allocation 



ingBandwidth maintains 603 a running average of band- 
width demand. The running average maintained by Blue- 
DotUsingBandwidth eliminates transient bandwidth spikes, 
by producing an average of bandwidth demand for each 
application currently demanding access to the network, and 
constantly or periodically determines the average number of 
bytes per second each application has attempted to send to 
the network for the past predetermined number of seconds. 

BlueDotUsingBandwidth may be implemented using the 
following code: 



_BlueDotUsingBaiidwidth( 

GlobalsHandle 

long fanout, 

long bw 



){ 



} 



GlobalsPtr gp - *gh; 

gp->customer = TcContiection; 

gp->baiidwidth = (bw + ^->lastBaiidwidth) / 2, 

gp->lastBandwidth + bw; 

gp->fanoiit = fanout; 

Update Values(gp->process->universals); 

ietum(noErr); 



BlueDotUsingBandwidth calls Update Values 605, which 
actually performs the steps of determining the values of the 
variables mentioned above (fan out/number of connections, 
number of applications, and total bandwidth being used). As 
implemented according to one embodiment of the present 
invention, UpdateValues is shown in FIG. 6. 

UpdateValues determines the number of conferencing 
applications that are running on the system. This is deter- 
mined by maintaining a table in which all conferencing 
applications provide data when the applications run the 
embodiment of the present invention. For each application, 
the number of connections (i.e., the fan out associated with 
the application) is inserted into the table, along with the 
bandwidth being used by each application (i.e., the data 
transmission rate multiplied by the fan out). All of these 



values are stored in the table each time any of the Blue Dot 
sent to the system by the administrator. In one embodiment, 40 processes are called, and the values are provided to Updat- 
the Blue Dot process includes the following "external" 
routines, which exchange data with the various applications 
that call the Blue Dot process: BlueDotUsingBandwidth, 
BlueDotGimmeBandwidth, and BlueDotQueryHappiness. 
Each of these will be discussed in detail, and one embodi- 45 
ment of each presented. 



BlueDotUsingBandwidth monitors bandwidth use, deter- 
mined by accumulation. In one embodiment, monitoring of 
bandwidth is performed by a data splitter that periodically 
calls BlueDotUsingBandwidth. The process BlueDotUsing- 50 
Bandwidth exchanges information with the lower-level soft- 
ware serving each application, calculates the bandwidth 
used, calculates the fan out (i.e., the number of copies of 
each packet that must be provided to the network or com- 
munication medium) by each application, and calculates the 55 
total number of conferencing applications participating with 
Blue Dot. Thus, BlueDotUsingBandwidth derives the three 
values corresponding to the three variables produced by the 
node. Each of these values is stored in a location where all 



e Values. The system-level total of these values taken over all 
applications are also provided to the network. 

UpdateValues operates across applications. UpdateValues 
includes a first 602 loop that counts the number of confer- 
encing applications. The number of applications is initially 
set at zero 604. Each process is counted 606, until the final 
application has been counted. Nested within loop 602, 
UpdateValues proceeds to second loop 608 that totals the 
number of conferencing connections and also totals the 
current bandwidths being used by all of the individual 
applications. The second loop begins at the first process 610, 
and then proceeding through the processes that are engaged 
in a teleconference, adding the fanout and the bandwidth to 
each value 612. The total number of connections is also kept. 

Once the last process has been tallied, UpdateValues 
proceeds to a subprocess that detects whether someone at a 
client (non-administrator) node has attempted to alter the 
bandwidth allocation. This is accomplished by detecting 
whether certain function sets have been manipulated such 



other apphcations on the system can find the values (and 60 that the function sets caimot be used by UpdateValues. 



thereby update the values further). A data-splitter monitors 
bandwidth use, and calls BlueDotUsingBandwidth periodi- 
cally. 

In one embodiment of the present invention, shown in 
FIG. 5 BlueDotUsiQgBandwidth records 601 the bandwidth 
used, in bytes per second, for each application on the node 
attempting to send data over the network. The BlueDotUs- 
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UpdateValues also includes a variable "hacker" that is used 
to determine whether a user of a system who does not have 
access to the permissions associated with network adminis- 
tration is nevertheless attempting to turn the BlueDot pro- 
cess off. In other words. Hacker becomes non-zero when 
tampering by someone other than the administrator is 
detected. When hacker is set, the system assigns a small 
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allowed bandwidth. Hacker determines whether a user at a 
particular node has attempted to override the bandwidth 
allocation set by the network administrator, and, if unautho- 
rized tampering with the value set by the ne^vo^k adminis» 
trator is detected 616, hacker sets a standard bandwidth 
allocation 618. Hacker allows the net\\'ork administrator to 
control the bandwidth used by a particular application. 
Hacker determines whether a user at a particular node has 
attempted to override the bandwidth allocation set by the 
network administrator, and, if unauthorized tampering with 
the value set by the network administrator is detected, 
hacker sets a standard bandwidth allocation. The standard 
allocation, in one embodiment, is much smaller than might 
otherwise be allocated. 

Update Values then stores the four variables where other 
applications can find them, using the ChokerSet processes 
620. The table also includes a predictive value of the number 
of bytes per second the application may be expected to 
attempt to send over the network over the next brief period 



variable described above (i.e., the current total bandwidth 
being used for all conferencing functions on each particular 
machine, the number of active applications that are capable 
of sending messages over the network in the immediate 
5 fixture, and the number of active connections), but also stores 
a minimum and maximum bandwidth that are desired by the 
application. Although these values are determined by the 
application, not by the embodiment of the present invention, 
their values are read by BluedotGimmeBandwidth. Blue- 
10 DotGimmeBandwidth examines the bandwidth demands of 
the applications running on the node, and the minimums and 
maximums bandwidth requests mentioned above with 
respect to the table maintained by Update Values. The mini- 
mum and maximum bandwidth requests are limited to 
15 predetermined values; if the minimum requested is negative 
or zero, then the minimum requested is reset to one 702. If 
the maximum requested is for some reason less than the 
minimum, then the Maximum is set to at least one greater 
than the minimum 704. These limits prevent certain types of 



of time, taking fan out into account. Thus, if an application *i. *• i r * K.^. rm. . . ^ 

.•..v«p;tPHtntnH . innnnhvt....c ^Lv.*;.. mathematical errors from occurnng at the node. Then, a total 



is expected to send a 10,000 byte per second communication 
over the network with a fan out of two without miilti -casting, 
the table would contain a value of 20,000. For example, 
update values might accumulate the total number of bytes 
each application has attempted to send over the network for 
the past five seconds, and, by dividing the total by five 
seconds, determines the average bandwidth use. 

One embodiment of UpdateVariables may be imple- 
mented as the following code: 



update Values ( 

UniversalBitsPtr 

){ 

GlobaisPtr 
ProcessStuffPtr 



ub 



ps; 

long connections, bandwidth, apps; 

connections = 0; 
bandwidth = 0; 
apps = 0; 

for(ps = (ProcessStufiPtr) ub->processes-qHead; ps; ps = 
(ProcessStuflEPtr) ps->qLink) { 
appsHH-; 

£oi(gp « (GlobaisPtr) ps->instances.qHead; gp; gp = 
(GlobaisPtr) ^->qLink) 

if(gp->customer == TcConnection) 



35 



a)->fanout; 

S)->bandwidtli * g]i->£anout; 

} 



connections h 



bandwidth h 



} 



} 



if(ub->ha(^er) 

ub->aUowed = kHfeenyBit; 

else{ 

EnterCodeResourceQ ; 
ChokerSetActiveApplications(apps) ; 
ChokerSetActLveConnections(comiections) ; 
ChokeiSetCunentBandwidth(bandwidth) ; 
ChokerGetAllowedBandwidth(&ub->alIowed) ; 
LeavcCodeResonrceQ ; 

ub->used " bandwidth; 



mmimum bandwidth request for an application is 
determined, by multiplying the bandwidth requested mini- 
mum by the fanout 710. This calculates the bandwidth 
reqmrements of the application to communicate with each 
25 recipient of the teleconferencing messages. Once the mini- 
mum and maximum bandwidth requested by each applica- 
tion is determined, BlueDotGimmeBandwidth calls 
UpdateAllocations 708, As explained below, UpdateAlloca- 
tions tries to allocate to each application an amount of 
30 bandwidth between the minimum and the maximum. If less 
than the minimum demanded by a program is available, 
however, then zero is assigned. Also, no application gets 
more than its requested maximum. Finally, each application 
receives an allocation described in a bandwidth per connec- 
tion 710. 

Software associated with an application (generally, a 
low-level program called by a conferencing application as 
part of the I/O routines of the application) can thus request 
an additional share of the bandwidth allocated by the admin- 
40 istrator. For example, a user changes operating parameters of 
a teleconferencing application by enlarging a window on a 
screen (thus increasing the number of pixels per video 
frame), or switching to a less effective codec, or transmitting 
sound in stereo instead of mono, then at a lower level 
45 software associated with the application requests additional 
bandwidth from a process mnning according to one embodi- 
ment of the present invention. The lower level software 
associated with the application provides the request by 
providing a minimum and a maximum bandwidth, and also 
by providing a fan out and a priority. The software associ- 
ated with the teleconferencing application provides a request 
to one embodiment of the present invention, upon determin- 
ing that the application will run more eflSciently with addi- 
tional bandwidth. 

BlueDotGimmeBandwidth may be implemented accord- 
ing to the following code: 
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In addition to BlueDotUsingBandwidth, the embodiment 
of the present invention described above also includes a 
routine BlueDotGimmeBandwidth, which is also called 
periodically. As implemented according to one embodiment 
of the present invention, BlueDotGimmeBandwidth is 
shown in FIG. 7. According to one embodiment of the 
present invention, each application not only stores the three 
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^lueDotGinuneBandwidth( 
GlobalsHandle 
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long 
long 
short 
short 

GlobaisPtr 
gp->customer = 



minBW, 

maxBW, 
priority, 
fan out 

gp = "gh; 

TcFIower; 
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} 



if(minBW <= 0) 

minBW - 1; 
if(maxBW <=mmBW) 

maxBW - minBW + 1; 
gp->minRequested = mmBW * fan out; 
gp->maxRequested = maxBW * fen out; 
gp->priority = priority; 
Up<lateAI]ocations(gp->process->universals) ; 
return(g)->aUocated / fan out) ; 



long as there are any processes that have received less then 
the stated maximum thereof and so long as there is unallo- 
cated bandwidth remaining, UpdateAllocations divides the 
total remaining bandwidth by the number of processes have 
5 received less then their maximum, and attempts to allocate 
to each process the pro-rate share of the remaining band- 
width up to the maximum for the process, until either ail 
processes have received their maximum or all bandwidth is 
exhausted. ALLOCATED 840, CONSUMERS 842, AVAIL- 
10 ABLE 844, and the QUOTA 846 vary, but provide allocation 
of the bandwidth among the other applications. The appli- 
cation is then informed of its new allocation. Although in 
this embodiment of the present invention, the class of 
applications selected for allocation by the administrator is 
15 the class called "conferencing applications," it will be appar- 
ent that other applications could have been selected. It will 
be apparent that the group of applications, or even other 
types of processes, could have been selected. All of these 
values are all stored in the table, and are all provided to 



After calling UpdateAllocations, BlueDotGimmeBand- 
width provides the maximum allocation of bandwidth per 
copy (taking fan out into consideration), for each process 
seeking transmission access to the network. 

UpdateAllocations resets the allocation of bandwidth of 
each process that sends data to the network or communica- 
tion medium. As implemented according to one embodiment 
of the present invention, UpdateAllocations is shown in 

FIGS, 8A, SB, and 8C. UpdateAllocations includes a first 20 Update Values as well as the networic. 
loop, that iterates through the processes seeking network 
transmission access and determines (1) the priority of each 
process, and (2) the total of all the allocations assigned by 
UpdateAllocations the last time UpdateAllocations was 
called. To determine the priority for each process and the 25 
tolal allocation, variables ALLOCATED and MAXPRIO are 
initially set to zero 802. Starting at the first process 804, 
UpdateAllocations iterates through each process, determin- 
ing whether the priority of the particular process is greater 
than MAXPRIO 806; "if so, the value of MAXPRIO is 
changed to priority of the process 808. Whether or not the 
particular process is of the highest priority, the bandwidth of 
the process is added to ALLOCATED 810. The iteration 
continues until the last process has been counted 812. 



UpdateAllocations may be implemented by the following code: 



UpdateAIlocations( 
UniveisalBitsPtr 

){ 

BlobalsPtr 
ProcessStuflEPtr 



ub 



8p; 
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allocated, available, quota; 
prio, maxprio, consumers; 



long 
short 

allocated = 0; 
maxprio = 0; 

for(ps ~ (ProcessStuffPtr) ub->processes.qHead; ps; ps = 
(ProcessStuffPtr) ps->qLink) 

for(gp = (GlobalsPtr ps->tnstances.qHead; gp; gp = 
The variables ALLOWED and USED are determined, and 35 (GlobalsPtr) g>->qLink) { 
ALLOCATED and AVAILABLE are calculated 820, When if(gp->customer != TcRower) 

continue^ 

unauthorized bandwidth users at the node are using if(gp->priority > maxprio) 

bandwidth, the AVAILABLE bandwidth is reduced. maxprio - 

If AVAILABLE is less than or equal to zero, then no >priority; 
bandwidth remains to allocate, and UpdateAllocations ends 40 
861. If bandwidth remains, however, then AVAILABLE is 
set to the maximum of AVAILABLE and ALLOWED 862. 

UpdateAllocations then assigns minimum bandwidth to 
the processes having the highest priority. Starting at the 
highest priority 822, UpdateAllocations identifies all the 45 
processes (starting with the first 824 ) having the highest 
priority and allocates to each the minimum bandwidth 
requested by the process 826. A running total AVAILABLE 
is reduced 828 by the bandwidth so allocated, the variable 



} 



allocated +=gp ->allocated; 
gp->aIlocated = 0; 



ps = 



consumeis « 0; 

available = ub->allowed + (allocated - ub->used); 
if(available <= 0) 
return; 

if(avai]able > ub->allowed) 

available = ub->alIowed; 
for(prio = maxprio; maxprio = 0, prio; prio = maxprio) 
for(ps = (ProccssStuffRr) ub-> process.qHead; ps; 
(ProcessStuflEPtr) sp>qlink) 

for(gp - (GlobalsPtr) ps- 



Storing the total amount of bandwidth that can be assigned 50 >instances.qHead; gj; gp = (GlobalsPtr) gp->qLink) 



within the limit imposed by the administrator. Each process 
at the given priority is assigned bandwidth so long as 
AVAILABLE is not zero 830, and then UpdateAllocations 
turns to the next-highest priority, etc. until each process has 
at least the minimum requested by the process. If possible, 55 
UpdateAllocations give at least the minimum bandwidth to 
each process. 

The number of applications that have received their 
minimum, is labeled the CONSUMERS. If there are con- 
sumers (i.e., appUcations that could receive an additional 60 
bandwidth allocation without exceeding their maximum) 
and bandwidth is available 832, then a quota is defined 834. 
The quota represents a small portion of bandwidth than can 
be reassigned. Starting at the first process that has received 
less than the maximum bandwidth requested (i.e., the first 65 
consumer) 836, UpdateAllocations proceeds through each 
process giving the process more bandwidth. Repeatedly, so 



{ 



>minRequested) { 

>allocated = gp->minRequested; 

available -=» gp->niiiiReqiiested; 

consumers-H-; 

if(av£iilable «» 0) 

return; 



if(^-priority = prio){ 
if(available : 



} 



}else if(gp->priority > maxprio) 
maxprio = gp- 

>priority; 

} 

whUe(consumeTS > 0) { 

quota = available / consumers; 

if(quota == 0) 

return; 

for(ps - (FrocessStuflfPtr) ub->processes.qHead; ps. 
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-continued process can use. If the amount allocated is less than the 

minimum, then the score is set to zero. The amount that the 



UpdateAIlocations may be implemented by the following code: prOCess can USe is determined in UpdateAllocations, 

T ^ , "TTr described below. If the amount allocated is equal to the 

ps = races u r; sp->q i ^^^^^ ^ (GiobalsPtr) ps- 5 niinimum, then the happiness score is 50%, if the amount 

>mstances.qHead; gp; gp - (GiobalsPtr) ^->Lmk) { allocated is less than the minimimi, then the happiness score 

if (:a3->a]located) is 0%, and if the amount allocated is equal to the maximum 

continue; (or greater), then the allocation is set to the maximum 

, if((gp->aUocated + quota) requested and the happiness score is 100%. 

>-gp->maxRequested) ( consumers--; lo BlueDotQueryHappiness provides information to the 

available applications. As illustrated with respect to update Values, 

(gp->maxRequested - gp->a]iocated) , Blue Dot includes subroutines that operate across 

gp- applications, and each application provides information to 

>allocated - gp->maxRequested; gl^e Do^. illustrated with respect to 

available -= BlueDotGimmeBandwidth, this information includes a 

quota; maximum bandwidth request for each network process 

gp. serving a conferencing application. 

>a]located += quota; The average of aU the scores is the happiness factor, as 

) illustrated in one implementation by the following code: 

if(available 0) 

return; 

I 20 

T ^lueDotQueryEIappiness( 

■> GlobalsHandle gh 

i ){ 

GiobalsPtr g^c = "gh, 

Nif Although not absolutely necessary for operation, several GiobalsPtr ^ gp; 

ipi; embodiments of the present invention also includes a routine 25 ^ " * 

law? T^i TT • X • 1 J J- . ProcessStuffPtr ps; 

.^.^ BlueDotQueryHappmess. As implemented accordmg to one ^^^^^ g^^^^. 

embodiment of the present invention, BlueDotQueryHappi- gx->customer = TcAppiication; 

nessis shown in FIG. 9. The routine BlueDotQueryHappi- ub = gx->process->umversals; 

jw ness monitors how the bandwidth allocated to any given =" 

Z! node is being parceled out among the conferencing appU- 30 score = 0; 

j-^: . 11 • -, , 1 . i 1 ior(gp = (GiobalsPtr) ps->processes->instances.qHead; gp; gp = 

*; cations and can also provide a visual mdicator of how the (GiobalsPtr) gp->qLiiik) 

-f 5 available bandwidth is being parceled out. This is accom- if(gp->customer == TcFlower){ 

, 5™ plished by calculating, in some embodiments, a "happiness" count++; 

factor for each appUcation, and in some embodiments abo ,„„„,^ 

. /'^ 1. • 1: r TM. score +» 50 + (50 • (gp->allocatea - 

^ CaiCUlatmg a global happiness factor for each machine. The 35 gp->mmRequested)) / (gp- >maxRequested - gp->minRequested); 

lir happiness factor can be visually presented to a user so the } 

= 3 user can see the amount of bandwidth assigned within the retuni(count ? score / count : 100); 

^ machine to each application, I 

'y-j To calculate the happiness factor, the process compares 

i.Q the current bandwidth being allocated to the process for 40 A happiness color may be assigned, such that the color is 

S conferencing functions with the maximum bandwidth red when the happiness factor is less than 0%, blue when the 

requested, and determines a first ratio. Thus, at each work- happiness factor is 100%, and purple when the happiness 

'r^ station or node, an "application happiness" factor corre- factor is between 0% and 100%. 

sponding to each application may be calculated, indicating The described embodiment of the present invention can- 
how much of the bandwidth desired by the application is 45 not force other applications to use less bandwidth, but can 
available to the application. As stated above, although in this provide feedback to the other applications by informing 
embodiment of the present invention the administrator them that there is some interference among applications at 
assigns a maximum bandwidth for conferencing the network interface. Thus, other applications may consult 
applications, it will be apparent that any group of BlueDotQueryHappiness to determine whether data gener- 
applications, or even a group of other types of processes, 50 ated by a particular application is reaching the network, and, 
could have been selected. if not, whether the reason is because higher priority or other 
In one embodiment, a visual embodiment of the happiness applications are usurping bandwidth allocation. In such an 
indicator is presented as a red dot when the corresponding embodiment, the application can switch to a lower band- 
happiness factor is 0 percent, a purple dot when the corre- width use, turn on additional data compression, send fewer 
sponding happiness factor is between 0 and 100 percent, and 55 frames, or otherwise reduce bandwidth use to allow other 
a blue dot when the corresponding happiness factor is 100 applications to proceed more efiBciently, or in the alternative 
percent. The happiness factors may be displayed as colored may increase bandwidth demand to gain greater access to 
dots on the screen of a monitor or connected to the machine. the network. Depending on the particular application, win- 
If the maximum bandwidth allocated to the machine by the dow size might be reduced, sound may be less complex, or 
administrator is greater than the total of all the maximums 60 additional users seeking access to the teleconference may be 
over all the processes trying to send data to the network, then rejected. 

all the processes are able to send data at their full data rate During a teleconference, additional users may enter the 

(bandwidth) and the "happiness" is 100%. teleconference and thereby increase the fan out. The low 

BlueDotQueryHappiness otherwise iterates through aU level flow control process detects the additional fan out, and 

the connections used by the application and assigns a score 65 provides the fan out update to BlueDotQueryHappiness. 

to each, the score being 50 points plus half the percentage of BlueDotQueryHappiness then updates the table maintained 

the difference between maximum and minimum that the by the blue dot process. Because multi-cast may not be 



